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ABSTRACT 

The purpose of this mixed-method study was to evaluate a Higher Education 
Administration doctoral program, to understand student perceptions of program quality, to 
improve both student satisfaction and retention to degree completion, and to plan for the future. 
Findings from survey data and focus group interviews indicated program strengths to include 
caring and supportive faculty, excellent teaching, and useful, real world application of 
curriculum content. Findings also indicated several program challenges including inconsistent 
leadership due to faculty and administrative turnover, inconsistent policies and practices, lack of 
proper student advising and mentoring, curriculum redundancy, unclear distinctions between 
degree programs, and lack of proper research preparation for the dissertation stage. The 
researchers prioritized program changes based upon findings where survey and focus group data 
intersected. Currently the Higher Education Administration program is making changes to its 
required curriculum in order to reduce hours to degree completion, to eliminate curriculum 
redundancy, and to clarify distinctions between the Ed.D. and Ph.D. programs. In conducting this 
program evaluation however, the researchers have realized that some program challenges may be 
systemic, and therefore, not easily remedied. 

Keywords: Mixed-methods, action research, Higher Education Administration, program 
evaluation, student perspectives 
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INTRODUCTION 

This paper is about the process the authors have undertaken, as teachers, researchers, and 
administrators, to evaluate the doctoral program in Higher Education Administration (HEA) at 
our institution, to improve program quality, and ultimately to increase both student satisfaction 
and retention. In taking steps to ask questions about the program in efforts to improve, the 
researchers have also inadvertently improved their own professional lives. As teachers, the 
researchers have become more aware of student needs; as researchers, the power of research to 
influence and change practice is now more evident; and as administrators, the authors are more 
aware of the power needed to influence change, even on a small scale. 

The Context for the Study 

As researchers, it is important to provide readers with context for the study as well as 
provide a larger context of the institution and how the department and program are situated 
within the study. For the past decade the doctoral program in Higher Education Administration at 
this institution has experienced continual faculty and administrative turnover, resulting in 
changes in policies, practices, teaching, and advising. Students took notice of these changes and 
vocalized their dissatisfaction with the lack of program continuity, lack of consistent advising, 
and lack of leadership. However, students were largely vocal only within the confines of their 
own student groups; occasionally these “grumblings” surfaced so that faculty and administrators 
would hear student complaints. It is important to note though, that among the complaints heard 
were also positive remarks about program strengths. The primary reason this action research 
study was done was to better understand what the program is doing well, where the challenges 
are, and ways to begin the change process. 

The Context of the Institution, Department, and Program 

The research university is housed on a lovely campus in the deep south amidst giant oak 
trees that shield the classic red brick and white columns of the campus buildings upholding true 
southern tradition, including a traditional top-down hierarchical governance structure. The 
Higher Education Administration program is situated within a larger department that houses 
educational foundations and research. This department resulted as a recent division of the 
original Department of Educational Leadership and Research. The Higher Education 
Administration program contains both specialist and doctoral degree and aims to provide 
students with a solid foundation in higher education coursework through required core courses, 
while also providing a solid foundation in research and statistics. 

Three degree programs are offered within the Higher Education Administration program: 
the specialist (Ed.S.), the doctor of education (Ed.D.), and the doctor of philosophy (Ph.D.). 

The specialist degree (Ed.S.) is approximately half the length of a doctoral degree, and allows the 
student the option of stopping at that point with a terminal degree or to count those credits 
toward one of the doctoral options. The two doctoral degree options (Ed.D. and Ph.D.) include 
areas of specialization in addition to required core courses. Areas of specialization include an 
approved outside minor in a subject area, a community college emphasis, cognates/electives, a 
research minor, a research specialist option, and certificates in Adult Education and Institutional 
Research. The Ed.D. program is suited for those whose interests lie in practical application and 
effective practice in a post-secondary setting. Students are encouraged to choose this degree plan 
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if their ultimate goal is to serve in an administrative position at a college or university. However, 
this degree option allows flexibility by preparing students to teach and conduct research in a 
college setting. The Ph.D. program in Higher Education Administration is primarily for those 
applicants whose interests lie in gaining a tenure-track position in a Higher Education 
department at a college or university. Furthermore, the Ph.D. program prepares students to 
conduct research both inside and outside of college/university settings. 

Currently, there are 68 students enrolled in the HEA Program at this southern university. 
On the faculty side, there are currently several tenure-track faculty members who teach courses 
in the HEA program, even though their areas of expertise lie outside of higher education. 

Faculty also includes one visiting assistant professor and four adjunct faculty members who have 
served in various professional capacities ranging from former college presidents to human 
resource experts. The program also has a coordinator serving a dual role as coordinator for the 
university’s Higher Education Administration program and the Adult Education program. 

PURPOSE OF THE STUDY 

The purpose of this program evaluation was to gather data to understand students’ 
perceptions of program quality and levels of satisfaction in the Higher Education Administration 
doctoral programs. As researchers, the primary goal is to use the data to both improve the quality 
of the HEA program and to plan for the future using a continuous cycle of improvement. 

Ultimate goals are to retain each student to completion of their Ph.D. or Ed.D., and to ensure 
each student has received a quality education in preparation for their respective careers. 

RELATED LITERATURE 

Literature on Graduate Student Retention and Satisfaction 

Throughout this nation’s history and especially during the past several decades federal, 
state and local governments, public and private sector employers, college faculty and 
administrators, and certainly many families, have realized the importance of higher education to 
the nation’s economic vitality. Stakeholders have responded by making higher education more 
accessible to the masses, including more access to both undergraduate and graduate programs. 
Although many believed that these attempts to make higher education more available to our 
nation’s citizens would result in more people earning advanced college degrees, this has not been 
the case (Seidman, 2005). Like any other enterprise, the more higher education grows and 
expands, the more problems it encounters—one of which is student attrition. As a result, 
students are prematurely leaving college before earning their degrees, thus leaving the students in 
a state of dissatisfaction because they did not achieve their academic, career, and social goals. 
This is especially true in graduate programs, where student success depends on their academic 
and social experiences, as well as their interactions with their faculty and department (Tinto, 
1993). 

Tinto reported (1993) that doctoral level completion rates were mediocre at best, 
estimating that nearly 50 % of those students who enter never graduate from their programs. 

From that report over ten years ago, there has been little change. Currently, the average number 
of students who complete their doctoral programs hovers around 55 % (Council of Graduate 
Schools, 2008). There are some institutions that have a higher than average graduation rate. The 
University of Maryland, for instance, conducted a research project of its own doctoral students 
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and found the graduation rate to be less than 40% during the ten year period from 1994 to 2004. 
Therefore, it is important that doctoral programs at institutions of higher learning constantly self- 
evaluate to ensure they are meeting the needs of their students (Seidman, 2005) and providing 
learning experiences that students believe are beneficial. When students are satisfied and feel that 
their needs are being met, they are more likely to succeed academically and persist to graduation 
(Pascarella and Terenzini, 2005). Tinto (1993) asserts that studying the social and academic 
experiences of graduate students is important when making program decisions and 
improvements, as well as developing institutional policies to help improve graduate program 
completion. 

Graduate Student Satisfaction 

Studies conducted throughout this decade sought to measure and understand the graduate 
student experience, in particular, the level of student satisfaction. A review of literature identifies 
several common and recurring themes regarding factors impacting level of satisfaction. These 
factors include, but are not limited to, being given the ability to make informed decisions; having 
mentoring opportunities that provide the opportunity to assess and evaluate one’s growth and 
performance as a graduate student; receiving quality advisement; being provided with career and 
professional development opportunities; being socialized into the academic department; 
understanding faculty work life; and being trained to teach at the college level. 

Fagen and Wells (2004) noted from their research that “satisfaction was strongly related 
to choice” reflecting a perception that decisions made by graduate students were often not 
respected by persons of authority (p. 79). Graduate students have indicated dissatisfaction with 
inability to make informed decisions prior to enrollment regarding program choice due to lack of 
consistent and up-to-date information on program expectations, graduates’ job placement rates, 
and time to degree (Gaff, 2002; Golde & Dore, 2001; Lovitts & Nelson, 2001; & Nyquist & 
Woodford, 2000). The lack of awareness of career options, both academic and non-academic, is 
a prevalent concern expressed in research findings. This lack of awareness has impacted 
students’ ability to make informed career decisions because of limited knowledge regarding 
differing expectations of future faculty at various institutional types in the areas of research, 
teaching, and service (Fagen & Wells, 2004; Nyquist, Austin, Sprague, & Wulff, 2001; & Wulff, 
Austin, Nyquist, & Sprague, 2004.) 

Mentoring issues have also consistently been identified as a major source of contention 
among graduate students. These issues include: the absence of a mentor altogether or a 
mismatch between student and mentor; a need for implementing a multiple mentor model 
wherein graduate students have the opportunity to interact with several faculty members 
throughout the duration of their program; peer mentoring with advanced students being paired 
with newly admitted graduate students; and having mentors who model ethical behavior (Gaff, 
2002; Golde & Dore, 2001; Fovitts & Nelson, 2001; & Nyquist et al., 2001). Graduate students 
look to mentors for professional development opportunities that provide exposure to the many 
facets of faculty work life (Austin, 2002) including typical roles and responsibilities, 
understanding of governance structures, and service expectations specific to the professoriate 
(Wulff, et al., 2004). 

The need for systematic and on-going advisement and feedback has been a major factor 
in determining the level of satisfaction among graduate students. Students desire initial 
orientations wherein consistent expectations and guidelines are set forth and followed. 
Advisement procedures that provide accurate and reliable information regarding curricula and 
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the managing of non-academic procedures prior to dissertation are particularly important 
(Lovitts, 2004). Throughout their journey, graduate students desire systematic feedback and 
evaluation which may include opportunities for self-reflection, formative evaluations such as 
annual reviews, and summative evaluations to more informally discuss their professional growth 
to identify areas of strength and areas for improvement (Austin, 2002; Nyquist & Woodford, 
2000; Wulffet al„ 2004). 

A lack of socialization and enculturation into the academic department are additional 
areas of concern regarding graduate student satisfaction, particularly for students enrolled on a 
part-time basis. Students have expressed a desire to be engaged with faculty and peers through 
both formal and informal social interactions. Building community affords students opportunities 
to understand and become engaged with their academic culture, especially at the departmental 
level (Lovitts & Nelson, 2001; & Nyquist & Woodford, 2000). Golde and Dore (2001) assert that 
socialization assists students in addressing the ambiguity of processes often found within a 
department and offers a venue for voicing concerns where students otherwise feel powerless to 
speak up. Engaging all students in common projects, research, teaching, and service 
opportunities are examples of means by which enculturation can be enhanced (Lovitts & Nelson, 
2001). If these two areas are neglected, student retention will likely be negatively affected as 
graduate students question whether or not they really belong in a professorial role and if the cost 
is worth the benefit both professionally and personally (Wulff et al., 2004). 

A final concern, expressed in the graduate student satisfaction literature, is not being 
trained to teach — particularly within research institutions where the value placed upon teaching 
is ambiguous at best. Training, if any, tends to come from faculty who hold students to the same 
expectations and expose them to the same experiences to which they were exposed as graduate 
students thus creating a cloning effect. Modeling what faculty members do in the classroom may 
not be meaningful training because this “teacher training” is often not grounded in learning 
theory or best practices in teaching and learning at the college level. Also, graduate students 
must be prepared to operate in a much broader and rapidly changing global workplace. By 
providing students with the opportunity to engage across disciplines and across institutional 
types, their perspectives, expectations, and experiences can be more realistically aligned (Wulff 
et al., 2004). 

Lovitts (2004) states that nationally “the data have shown that 50 percent of people who 
start doctoral programs leave without completing the Ph.D.” (p. 115). This rate of attrition is 
alarming and should be cause for concern. Lrom her research, Lovitts offers the following 
conclusion: it is not what students bring with them to the doctoral program that impacts success, 
but rather what happens to the students while they are in their programs. The purpose of this 
research project has been to determine which, if any, of the graduate student satisfaction factors 
identified from the research are impacting program retention as well as the level of satisfaction 
among students participating in our program. 

METHODOLOGY 

An overall action research methodology (McNiff & Whitehead, 2006) served as a way to 
first observe and listen to what students were saying about the program and then to reflect on 
what we could do from a teaching, research, and administrative capacity. A typical action 
research methodology adheres to the following cycle: 1) researchers first observe what is 
occurring in their environment; 2) researchers reflect on observations and consider ways to move 
forward to make needed changes; 3) actions are taken to gather data for decision-making; 4) 
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environment is evaluated based on data findings; 5) modifications are made to the environment; 
6) researchers move in new directions and plan for the future. Because methodological design is 
considered a cycle of research, evaluation and planning are ongoing and the cycle of research is 
repeated as needed. 

This research study used a mixed-method design, collecting and analyzing descriptive, 
quantitative and qualitative data from current students who have been in the Higher Education 
Administration Program at least two semesters. This was accomplished through a survey 
instrument and focus group interviews; focus groups were used to understand how students 
described their experiences in the doctoral program and how they characterized factors that lead 
to satisfaction and retention. As researchers, we sought to capture as much information as 
possible about students’ perceptions of the program and also their experiences within the 
program and the meanings they were assigning to these experiences. Our goal in mixed method 
design was to find data intersections between the two collection methods. We determined that 
where study findings intersected between the quantitative and qualitative data, these would be 
the areas of focus for program improvement and planning for the future. 

The researchers’ target population for this study was the entire student population of the 
Higher Education Administration graduate program who had completed at least two semesters of 
coursework. The researchers felt that students who had completed at least two semesters in the 
program would have enough experience to provide meaningful, accurate feedback. This 
population consisted of 68 students. The Higher Education Administration student population 
primarily consists of Caucasian and African American students, with the majority being females 
(approximately 63%). 

During the Spring 2008 semester, a locally designed survey instrument was used to 
obtain student perceptions of how the program met student needs regarding academic advising, 
helped students achieve educational and career goals, demonstrated a commitment to academic 
excellence, and prepared them to conduct research in a college/university setting. The primary 
researcher developed the survey instrument with the assistance of instructors in the Higher 
Education Administration program based on informal comments and concerns shared by 
students. In addition to student feedback and guidance from instructors, the primary researcher 
referred to related literature in the area of student satisfaction and persistence when developing 
the survey instrument. Before being sent to students, the instructors met to provide their 
feedback and suggestions for improvement. Once the survey instrument was finalized, it was 
placed online through the University’s Office of Institutional Effectiveness online surveying tool, 
and emails containing the survey link were sent to all students in the HEA program who had 
completed at least two semesters. The survey remained active for approximately one month (late 
March to late April). The email explained to students why they received an invitation to 
participate in the survey, the expected amount of time to complete the instrument (10-12 
minutes), and an informed consent statement. The students were also assured of the anonymity of 
their responses to encourage their honesty and frankness. In addition to the email, faculty 
informed students about the purpose of the survey during class meetings and reminded them to 
complete the survey online. At the end of April 2008, the data obtained by the online surveying 
tool was exported directly to SPSS for analysis. 

The items measuring student satisfaction with academic advising, obtainment of 
education and career goals, program commitment to academic excellence, and preparation to 
conduct research in a college/university setting were measured on a five-point Likert scale 
(Strongly Agree, Agree, Neither Agree nor Disagree, Disagree, Strongly Disagree), where higher 
responses indicated higher levels of agreement. There were nine items measuring the academic 
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advising construct, seven items measuring preparation to achieve career and education goals 
construct, nine items measuring the program’s commitment to academic excellence, and six 
items measuring student preparation to conduct research construct. The researchers conducted a 
reliability analysis of the constructs, which were .897 for academic advising, .940 for preparation 
to achieve career and educational goals, .890 for the program’s commitment to academic 
excellence, and .932 for preparation to conduct research. Since all of these reliabilities were 
greater than .700, the instrument was considered to produce reliable scores. 

As a follow up to the survey data, the researchers collected qualitative data through focus 
group interviews during the Summer 2009 semester. Four students nearing the end of their 
coursework in the Higher Education Administration program agreed to serve as focus group 
facilitators. After receiving training from the primary researcher, each of the four students 
facilitated one focus group interview comprised of four or five students. The primary researcher 
trained the facilitators by interviewing them in a focus group, following the exact protocol 
(Appendix A) and procedures they would use when conducting their focus groups. Since the 
interview protocol was relatively short and simple, this training session also served as a focus 
group interview of these four students. 

To schedule the focus groups, the primary researcher obtained a list of students in the 
Higher Education Administration program who had been actively enrolled for two or more 
semesters. Focus groups were scheduled by contacting students on the list; selecting students to 
participate in the focus groups was largely done through convenience. Although the focus groups 
were considered a sample of convenience they were representative of the HEA student 
population as they consisted of males and females, Caucasian, and African American students 
with a balanced proportion of each group. The focus groups were held during the month of June, 
at times conducive to the participants’ schedules. At the beginning of each focus group the 
facilitator explained the purpose of the focus group interview, data confidentiality, the 
approximate time required (one hour), and the informed consent process. The focus group 
facilitators followed the interview protocol while recording their sessions. All focus group 
interviews were recorded with a digital voice recorder, and the data were downloaded to the 
primary researcher’s computer for analysis where it was password protected. 

Recorded focus group interviews, each approximately one hour in length, were analyzed 
collaboratively by the four lead interviewers. The lead interviewers, as a group, listened to each 
focus group recording three times, paying attention to reoccurring themes in each focus group 
and across focus groups, making note of these themes. Lead interviewers had to agree upon 
emerging themes in order to provide internal validity to the findings. Participants’ key words and 
phrases were noted as supporting evidence for emerging themes. Quoted segments from different 
participants were then chosen as supporting evidence of the broader themes. Emergent themes 
were then categorized based on commonalities of underlying issues. Elsing the constant 
comparative method (Strauss & Corbin, 1990) of data analysis, the interviewers moved back and 
forth among the data categories, collapsing them until categories were set. The lead interviewers 
also followed typical case study analysis (Creswell, 1998); they developed the issues of the case 
as they emerged from the data and provided detail though key quotes that highlighted the larger 
issues. 
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FINDINGS 

Quantitative 

Thirty-four students between the ages of 27 and 56 (with a mean age of 36.4) responded 
to the online invitation to participate in this study. The majority of respondents were Caucasian 
females. Most of the respondents were part-time students, and most students commuted to the 
university from a short distance. Overall, the sample obtained was representative of the target 
population. The questionnaire requested the following demographic information: gender, 
ethnicity, age, degree type, semester hours completed toward degree, commute length, current 
GPA, and current student status. See Table 1 for detailed demographic information. 

The items measuring student satisfaction with academic advising, obtainment of 
education and career goals, program commitment to academic excellence, and preparation to 
conduct research in a college/university setting were collapsed into a single, overall mean. These 
overall means for each construct are reported in Table 2. 

As evidenced by these descriptive statistics, students in the program indicated that they 
are not being prepared to conduct research in a university setting. However, the students are 
relatively satisfied in all other areas. 

All of the items measuring each of the constructs on the survey instrument can be found 
in Appendix B. Regarding the academic advising construct, two particular areas were of concern. 
Students indicated concerns with understanding the dissertation process and comprehensive 
exam expectations and requirements when compared to the other areas. Also, students expressed 
that they do not have a clear understanding of the differences between degree options (Ph.D. vs. 
Ed.D.). All items measuring how well the program is preparing students to achieve their career 
and educational goals were high, with only one exception. Students pointed out that they do not 
think the program offers enough opportunities to gain teaching experience. Items measuring the 
program’s commitment to academic excellence showed two areas of concern. First, the analysis 
revealed that students revealed that there are not enough courses offered each semester and that 
more courses should be offered in alternative formats (online, hybrid, executive, etc.). Also, 
students expressed that more teaching/graduate assistantships were needed. Each of these 
findings aligns with the graduate student satisfaction literature with regards to having the ability 
and information to make informed decisions about degree options, career choices, and program 
expectations (Fagan & Wells, 2004; Gaff, 2002; Golde & Dore, 2001; Lovitts & Nelson, 2001; 
Nyquist, Austin, Sprague, & Wuff, 2001; & Wuff, Austin, Nyquist, & Sprague, 2004). Student 
responses also echo the work of Lovitts, 2004, in that a lack of consistent, systemic advisement 
leads to a greater level of student dissatisfaction with their graduate program. 

Items measuring how well the program is preparing students to conduct research showed 
problems in all areas. All items had means below 3.00, and most means were lower than 2.50. 
The lowest item, which had a mean of 2.06, indicated that students do not feel they are being 
prepared to read, analyze, and critique current research in higher education. Near the end of the 
questionnaire, students were asked to indicate their overall satisfaction with the higher education 
program. Most of the students rated the program high, giving it a score of 4 or 5. The mean for 
the overall satisfaction question was 4.03 (on a 5-point scale), which indicates that students, in 
general, are satisfied with the program. One of the last items asked students if they would 
recommend the program to someone who is considering graduate school in the area of higher 
education. Nearly 80% of the students indicated they would recommend the program. When 
asked a follow-up question as to why they would recommend the program to others, the 
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overwhelming majority of students reiterated that the faculty are incredibly supportive and 
helpful. 

Qualitative 

Twenty students responded to the invitation to participate in the focus groups. These 
twenty students were divided into groups of five where a trained facilitator led the discussions 
according to the focus-group protocol (Appendix A). Like the survey results, the focus group 
participants were representative of the program’s population, consisting of 13 females and 6 
male participants, all of which were of either Caucasian or African American ethnicity. The 
results of the focus group interviews are discussed in the following paragraphs. 

Program Strengths 

The results from the focus group interviews correspond with the results from the survey, 
especially concerning the program’s strengths. The majority of students perceived that faculty in 
the program genuinely care about them and have their best interests in mind. This positive 
finding is a reflection of departmental faculty efforts to socialize and inculturate students into the 
academic department by engaging students through formal and informal social interactions to 
build a sense of community (Lovitts & Nelson, 2001; & Nyquist & Woodford, 2000). One 
student commented that “the faculty in this program are so supportive and go out of their way to 
help you.” Students also indicated that they believe faculty do an outstanding job of making 
course content applicable to real-world higher education situations. Students said they can see 
how their coursework is linked to what is occurring in the real-world of higher education and 
believe they will be able to use what they have learned in their current/future occupations. Also, 
a majority of students in the focus groups reported that the adjunct instructors (consisting of 
former college presidents and current professionals working on campus) are excellent teachers 
and are able to bring priceless experience to the classroom. One student commented, “Having a 
former university president and community college president as instructors is well worth my 
tuition money. I can’t think of a better way to spend it.” Another student added, “The amount of 
experience these (adjunct) instructors bring to the classroom is immeasurable. They are able to 
bring a wealth of wisdom and experience to the classroom that other professors cannot.” 

Program Challenges 

Students in the focus groups pointed out several weaknesses and areas of improvement 
for the Higher Education Administration program. First, the students are very concerned about 
the program’s leadership, noting the program, department, and college have had several 
leadership changes during the past five years or more. One student commented, “Thank 
goodness for the few committed faculty and adjuncts we have. If it weren’t for them, we’d have 
no clue as to what is going on.” Another student said that “the leadership in the department has 
been playing musical chairs for the past few years—it will be nice when they finally make up 
their minds about who belongs where.” 

Students also had concerns about the program’s 78-hour curriculum being too 
cumbersome, including coursework seen as redundant and overlapping content areas. One 
student claimed, “The program is too long. If I didn’t live so close to here, I would have 
definitely gone to school elsewhere.” Other students expressed that many of the courses are 
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repetitive and can be combined into fewer courses. A handful of students complained that not 
enough courses are being offered in variable formats. One student noted, “This program touted 
its flexible course offerings when recruiting me. I feel like I have been cheated.” While the 
majority of students indicated they are satisfied with the formats of online, hybrid, and other 
alternative course offerings, they said there are simply not enough of these course formats 
offered on a consistent basis. Again, these findings substantiate the literature regarding students’ 
ability to make informed decisions, specifically regarding time-to-degree issues and flexibility of 
program format information (Lovitts & Nelson, 2004). 

Several students voiced their concerns about the program’s direction. Specifically, 
students expressed concerned that the program is turning toward a more faculty-preparation, 
theory-laden, research program. One student stated, 

It seems that those in charge have forgotten that this is a Higher Education 
ADMINISTRATION (student emphasis) Program. Since most of our students 
are administrators here or at other local community colleges, the administrative 
side of higher education should be the focus of the program. Currently, the 
only focus on administration from the courses comes from the adjuncts. I 
think the other faculty have forgotten that this program should be preparing 
administrators for practice. 

Other students concurred with this statement with the majority adding that they do not desire to 
become professors or researchers. Other students mentioned that a research minor is available in 
the current degree choices, and those interested in pursuing a career in research can take those 
courses if desired. 

The most frequently discussed program weakness, by far, was the number of required 
research courses. A clear majority of students do not feel like they are adequately learning 
important research skills that they will be able to use in their work lives or for their dissertation 
research. Many of the students said what they learn in their research courses is a lot of 
information with little context. One student said, “The research courses have zero benefit. 

None. All we do is learn about t-tests, ANOVAs and every other statistical procedure the mind 
can imagine. We have computers and software that can take care of that. Shouldn’t we be 
primarily focused on HOW to conduct research in a college setting?” In each focus group 
session, that same idea was expressed. As a result, many students said they do not feel like they 
will be ready to carry out their dissertation studies. Furthermore, three students currently 
working on their dissertations said that it is a very frustrating experience because they had never 
been taught how to conduct research. In summary, most students feel that the research courses 
are lacking a methodology and design component which is not addressed until the final semester 
of coursework. 

Finally, students who participated in the focus group interviews identified several areas 
for improvement. To improve the research process, students suggested that faculty could help 
students choose a dissertation topic early in the program, and every course could have a research 
design component (especially the research courses). As noted in the literature, students want 
assistance with non-academic procedures, particularly those involved in the dissertation process 
(Lovitts, 2004). Students also commented that the best statistics instructor teaches the only 
methodology course at the end of coursework, and they suggested that the class should be 
offered sooner during the program so students can gain an understanding of how to design and 
conduct research. 

Other suggestions included condensing the bloated curriculum into fewer courses, thus 
eliminating content redundancy or overlap. Specifically, students said that the higher 
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education’s law and human resources courses could be condensed into one course because both 
overlap content, as well the current vocational and career technical courses. The suggestion was 
also made by students (especially veteran students) to condense/change some of the research 
courses from learning how to calculate statistics by hand, to learning how to carry out the 
research process. 

Students also discussed the need to make the differences between degree options clearer 
(Ph.D. vs. Ed.D.). One very frustrated student stated that she has finished her coursework for the 
Ed.D. degree option and has taken more research courses than some students who are earning 
Ph.D.’s. She said this is unfair because it is perceived that those who earn Ed.D.’s do not receive 
as much research preparation as those who earn Ph.D.’s. Other students suggested that a clear 
distinction should be made between the two degree options. For students whose career goals are 
primarily administrative in nature, the Ed. D. option should be selected; for students who wish to 
pursue a faculty position with expectations of research, the Ph.D. options is the more appropriate 
choice. Almost all of the groups said a clear distinction such as the one stated above would likely 
alleviate confusion regarding the degree choices. 

While students in general were satisfied with flexible course formats, several said that 
more flexible course offerings should be made available, especially since the program touts this 
as a strength. Also, when presented choices of numerous course formats, students (according to 
the survey results) preferred having courses offered in the following formats ranked in this order: 
night courses, hybrid (partially online and partially face-to-face), and Saturdays. The students in 
the focus groups said the program is currently offering enough night classes, but more courses 
should be offered in hybrid format, online, and on Saturdays. The overall consensus of the 
students was that many of them are already working full-time, and they need course offerings 
that are conducive to their schedules. While they believe that the program is currently doing this 
for the most part, they also believe there is room for improvement. 

Lastly, the groups discussed the benefit of more orientation sessions to help new students 
become assimilated in the program and college. Many students said that they “learned the ropes” 
from current and/or former students, and orientation would be an ideal place for new students to 
be acquainted with seasoned students, faculty, and staff. Another suggestion included assigning 
new students to a peer mentor, who is an experienced student in the program. This peer-mentor 
student would be responsible for helping that new student become familiar with processes and 
procedures during the first year, like how to get registered for classes, etc. The need for peer 
mentoring by advanced students is supported in the literature (Gaff, 2002; Golde & Dore, 2001; 
Lovitts & Nelson, 2001; & Nyquist & Woodford, 2000). 

Data Intersection of Focus Group and Survey Findings 

One of the benefits of conducting a mixed-method study is finding where the data 
intersect and/or overlap. In this specific case we used the points where data intersected from 
survey and focus group findings to hone in on specific areas of program improvement. Based on 
survey and focus group findings the students reported four specific program strengths: 1) Faculty 
are valued for showing care and support for students; 2) Learning experiences are seen as 
valuable and have real-world application; 3) Adjunct faculty are viewed as excellent instructors 
and valued for the real-world experience they bring to the classroom setting; and 4) Students, in 
general, like the variable course formats offered. 

Based on survey and focus group findings the students reported seven areas where the 
program faces challenges: 1) Lack of consistent leadership; 2) Overlapping and redundant 
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curriculum; 3) More courses need to be offered in alternative/flexible formats; 4) More focus 
needs to be placed on preparing administrators in higher education settings; 5) Students require 
stronger preparation in the research process and research design in preparation for the 
dissertation stage; 6) Students need support/guidance early in their doctoral career for choosing a 
dissertation topic; and 7) The distinction between the Ph.D. and the Ed.D. needs to be clarified. 

DISCUSSION 

In general, the results of the survey and focus groups coincide; data overlapped in several 
areas and pointed out where program areas were weakest and most in need of improvement.. 
Survey results indicated students had a reasonably high level of overall satisfaction with the 
program, which concurs with what students reported during the focus group interviews. Based on 
this finding one may assume that if overall satisfaction is high and student needs are being met 
this will positively impact student success and doctoral retention as suggested by Tinto (1993) 
and Pascarella and Terenzini (2005). Specifically students discussed their overall satisfaction 
with their relationships with faculty members in and out of the classroom. This is an especially 
important finding since it closely aligns with findings in the research literature regarding 
importance of faculty/student engagement (Tinto, 1993; Pascarella & Terenzini, 2005; Lovitts & 
Nelson, 2001; Nyquist & Woodford, 2000; Golde & Dore, 2001; Wulff et al., 2004). However, it 
is important to note that while literature clearly addresses the importance of faculty and student 
mentoring relationships, our data did not reveal this as a significant area of our program in need 
of change. It is possible that because the majority of our students are preparing for administrative 
roles in higher education they may have already developed mentoring relationships with those 
they work with. It is also possible that students overall are satisfied with mentoring relationships 
with faculty, so this was not an important point of discussion for them. Overall, the majority of 
the students reported that they would recommend the Higher Education Administration program 
to other students. 

Survey and focus group results indicated that the program is not without issues and 
challenges. While most students indicated they would recommend the program to others, 
students consistently voiced concern about the following: 1) the high level of faculty and 
administrator turnover in the program, department, and college, 2) redundant curriculum, and 3) 
availability of additional flexible course formats. Their concerns are directly related to findings 
in the literature (Fagen & Wells, 2004; Gaff, 2002; Golde & Dore, 2001; Lovitts & Nelson, 

2001; Nyquist & Woodford, 200) that suggest students need to be able to make important 
choices based on informed decision-making. Faculty and administrative turnover negatively 
impacts informed decision making due to a lack of consistent advising and up-to-date 
information on program policies, expectations, job market concerns, and expected time- to- 
degree completion. Students’ time- to- degree is specifically and negatively impacted by faculty 
turnover because students often have to search for a new dissertation chairperson to guide them 
through the final research process. This problem alone may have a negative impact on doctoral 
student attrition. Student attrition is also impacted by time- to- degree, and a piece of this puzzle 
is reducing curriculum requirements to eliminate and/or combine those courses that clearly 
overlap in their curriculum design and coverage. And finally, informed decision making and 
time- to- degree may be negatively impacted by lack of proper information about available 
course formats since flexible course formats are especially essential to those students who may 
be working while they complete their doctoral degrees. 
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The survey and focus group results also indicated that students desire more assistance 
with research and dissertation preparation and more opportunities to gain teaching experience. 
While on the surface these may appear as separate issues they are both considered aspects of the 
socialization process - socialization into the department and also into the profession. Engaging 
students in common projects and research, teaching and service opportunities are all ways in 
which enculturation and socialization may be enhanced (Lovitts & Nelson, 2001). Engagement 
leads to greater student satisfaction and if these areas are neglected students are less likely to be 
retained to doctoral completion (Wulff et al., 2004). 

LIMITATIONS 

This research study has several limitations. First, the study is limited to a single program 
in Higher Education Administration at one university in the South. Second, the survey 
instrument had a 50% response rate. While this is a good response rate, a higher response rate is 
necessary to obtain the most accurate results when a small population is being studied. 

Therefore, a higher response rate may have yielded differences in program findings. The same 
holds true with the qualitative findings. Had more students been interviewed, findings could 
have possibly been different. As with any case-study evaluation done within a program or 
department by teachers and students, researcher bias might have been an issue even though 
precautions were taken to mitigate the likelihood of that occurring. 

LESSONS LEARNED 

As professionals, the researchers have learned four important lessons from conducting 
our program evaluation. First, the researchers have become more aware of the power of research 
to influence and change practices in our learning community. More specifically, they have 
learned that a mixed-method research approach yields richer data; researchers are able to 
consider a problem from more than one angle and able to examine where data overlap. Second, 
as professors, it is vital to become more aware of student needs and to meet those needs both in 
and outside of the classroom. Third, as administrators, it is also important to become more aware 
of the power to influence change, even on a small scale. Small changes may lead to improved 
student satisfaction, which in turn may improve overall retention. And finally, as members of an 
educational community, the researchers have recognized the importance of engaging in open 
dialogue with students because it reflects genuine concern and interest in program improvement. 
Students need to know their concerns are heard and that they also have the power to influence 
change in their own program. 

CONCLUSION 

While overall students reported satisfaction with the program, we have begun to address 
those program areas that need improvement. Currently our program took the necessary steps to 
reduce the number of required hours from 78 to 69 for the Ph.D. and from 72 to 66 for the Ed.D 
and a clear distinction was made between curriculum in the Ph.D. and the Ed.D. The Ph.D. now 
has a heavier research component, with a focus on less applied research along with core courses 
that are geared toward research and faculty preparation. And, students who voice an interest in 
preparing for a future faculty role are being matched with interested faculty in order to gain 
classroom teaching experience at the graduate level. The research tools for the Ed.D now have a 
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clearly applied focus and the core courses are geared toward preparation for leadership in 
administration; this addressed student concerns that our program was not providing enough 
preparation for higher education administrative positions. These changes were recently approved 
at the college level and at the graduate council. Beginning in Fall 2010 entering students to the 
Higher Education Administration program will be advised under the new plans of study. Faculty 
members have also begun discussions over providing more flexible course offerings in a possible 
weekend, executive format. However, this is under careful consideration, as the faculty and 
administration do not want to duplicate other programs in the region. More research is needed to 
determine if an executive weekend format would meet student need and to determine if it is a 
viable option. 

In addition to recent curriculum changes, a key change in the leadership structure is likely 
to bring longer-term stability to program leadership. In the past decade the Higher Education 
Administration program has operated under the direction and leadership of a Program 
Coordinator, however there was a great deal of turnover in this position leading to inconsistency 
in student advising, teaching, and program policies. One of the main reasons behind the faculty 
turnover in this position was likely due to the hiring practice of bringing in untenured, 
unseasoned faculty to coordinate the program. Once the junior faculty discovered the magnitude 
of responsibility, in addition to all of the other requirements to earn tenure, they typically left the 
university and found a faculty job elsewhere. Now, the Higher Education Administration 
Program Coordinator, is a seasoned, tenured, faculty member, and likely to remain in the 
position for a long term. This will bring consistency to program recruitment and admissions, 
advising and mentoring, teaching, and program policies in general. 

One area that has encountered roadblocks in the change process is addressing student 
concerns about their preparedness to conduct research, along with concerns for choosing a 
dissertation topic earlier in their coursework. Research courses are taught by the institution’s 
Research and Foundations program; however, when these faculty were approached to discuss 
several of the research findings, they were hesitant and resistant. At this juncture, this is one 
concern that needs further addressing, but this will only come about as program faculty are able 
to have open dialogue with faculty who teach research and foundations courses. This resistance 
could stem from the fact that faculty members are facing serious time constraints; budget cuts 
have required faculty to teach overload sections. It is also possible that this is a political issue 
embedded with other concerns yet unaware to others. 

Overall, the program evaluation begun in Spring 2008 has catalyzed needed changes in 
the university’s Higher Education Administration graduate programs. The researchers have taken 
appropriate, and measured steps to make key improvements to the program that will benefit 
students, faculty, and administrators in the department. And, following an action research cycle 
has proven to be effective for leading change. In this process, the researchers did the following: 
1) observed the environment, 2) reflected on what was observed, 3) gathered information, 4) 
evaluated our information, 5) made needed changes, and 6) are now reflecting on the “next 
steps”. 

While this study was essentially a case study of our own programs in Higher Education 
Administration, and thus findings cannot be widely generalized, there are several 
recommendations for other programs that follow from this study. First, a study such as this can 
only be effective if faculty and administration are willing to turn a reflective lens on their own 
practices and programs and if they are willing to hear, and understand, what their program 
strengths and challenges are. From a departmental faculty viewpoint, it was affirming to find out 
what the program was doing well from a student perspective, but it was difficult (at times) to 
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learn about the issues that clearly needed addressing. Second, faculty and administration need to 
be willing to follow through on findings and make changes as needed. This is more difficult than 
it sounds because it requires time, effort, and political finesse. In some cases changes may not be 
easy to instigate because faculty are already burdened by time constraints, and there may be 
political turmoil, turf battles, or budget constraints. Third, using an action research cycle proved 
effective as it provided a framework to observe and listen, gather data, make needed changes, 
and start the cycle again. Other programs may benefit from following the same type of action 
research method because it can be geared towards the needs of a specific educational 
community. And finally, getting students involved in the research process and really listening to 
their needs and concerns opened important dialogue, provided a shared sense of community, a 
commitment to change, and modeled the power of effective research to change practice. 
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TABLES AND APPENDICES 

Table 1 

Frequencies and Percen tages of Demographic Variables _ 

Variable Frequency Percentage 


Gender 


Male 12 35.3 

Female 22 64.7 

Race 

Caucasian 24 70.6 

African American 10 29.4 

Degree Type 

Ed.S. 4 11.8 

Ed.D. 4 11.8 

Ph.D. 24 76.4 

Semester Hours Completed 

Less than 18 hours 10 29.4 

18-30 hours 2 5.9 

33 - 45 hours 3 8.8 

48 - 60 hours 5 14.7 

More than 60 hours 3 8.8 

ABD (All but Dissertation) 5 14.7 

Did Not Respond 5 5.9 

Current GPA in Program 

3.00-3.24 3 8.8 

3.25-3.49 2 5.9 

3.50-3.74 2 5.9 

3.75-4.00 23 67.6 

Did Not Respond 3 8.8 

Commute Distance 

Live on Campus 3 8.8 

Less than 20 miles 18 52.9 

21-40 miles 5 14.7 

41-70 miles 2 5.9 

71-100 miles 3 8.8 

More than 100 miles 3 8.8 
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Student Status 

Full-time 11 32.4 

Part-time 23 67.6 


Table 2 

Satisfaction with Program Aspects 


Construct 

n 

Mean 

SD 

Academic Advising 

34 

3.73 

0.90 

Education/Career Goals 

34 

3.86 

0.76 

Academic Excellence 

34 

3.79 

0.66 

Research 

33 

2.43 

0.90 


Scale: 1 = Strongly Disagree...5 = Strongly Agree 


Appendix A 

Oral Presentation to Focus Group 

Each focus group facilitator will be required to follow the Interview Protocol verbatim. The 
expected time for focus group participation is 60 - 75 minutes. The facilitator will read the 
following statement at the beginning of the interview: 

“We will be asking you the following questions. (Read each question orally.) We anticipate that 
each question should take approximately 20 minutes to answer.” 

Interview Protocol 

If you had to rate the Higher Education Administration program on a scale of 1-10, how would 
you rate the program overall (1 being lowest, 10 being highest)? Provide a rationale. 

Following up on your response to question one, from your experience, (a) what do you believe 
are the greatest strengths of the Higher Education Administration program? Explain why; (b) 
what do you believe are the greatest challenges facing the Higher Education Administration 
program? Explain why. 

If you have specific ideas for program improvement, please share them. 
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Appendix B 

Descriptive Statistics for Academic Advising 



N 

Minimum Maximum 

Mean 

SD 

Upon entering the Higher Education program, I was 

34 

'X 


4.62 

.604 

assigned an advisor to meet with regularly. 



I was made aware of program requirements and 
expectations through various venues (program 

34 

'X 


4.71 

.579 

orientation, class meetings, discussions with faculty, 
advisement, etc). 


•D 




When advising me about education and career 

34 

a 

< 

4.91 

.379 

choices, my advisor has my best interests in mind. 

D 

J 

My advisor has discussed the career possibilities that 

34 

'X 


4.68 

.535 

a degree in higher education makes possible. 

J 


Requirements and expectations regarding 
comprehensive exams have been communicated to me 

34 

'X 


4.59 

.609 

through various venues (website, classroom 
discussions/lessons, advising, etc.). 






Faculty are approachable and available during office 
hours. 

34 

3 

5 

4.91 

.379 

My advisor is knowledgeable about the requirements 
of my degree plan. 

34 

3 

5 

4.85 

.500 

The differences between degree options was made 
clear to me so I coud choose which one is the best fit 
for my goals. 

33 

3 

5 

4.70 

.585 

Requirements and expectations regarding the 
dissertation have been communicated to me through 

34 

Q 

< 

4.62 

.604 

various venues (classroom discussions/lessons, 
advising, etc.). 

D 





Scale: 1 = Strongly Disagree ... 5 = Strongly Agree 






Descriptive Statistics for Academic and Career Preparation 





N 

Minimum 

Maximum 

Mean 

SD 

Faculty create learning experiences that do/will 
apply to my career in higher education. 


1 

C 

4.09 

.965 
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The Higher Education program is adequately 
preparing me to serve in an administrative 
capacity at a college /university. 

The Higher Education program is adequately 
preparing me to serve as a faculty member at a 
college /university. 

The program provides opportunities to gain 
teaching experience through teaching 
assistantships and/or team teaching with 
faculty. 

The Higher Education program is doing an 
excellent job of preparing me to write 
academically and professionally. 

The Higher Education program promotes a 
collaborative learning environment, allowing 
me to share ideas and learn together with other 
students. 

The program has prepared me to be an effective 
leader in higher education. 


34 1 5 3.88 1.094 


34 1 5 3.88 1.066 


34 1 5 3.15 1.209 


1 5 4.00 .985 


34 1 5 4.21 .880 


34 1 5 3.91 1.026 


Scale: 1 = Strongly Disagree ... 5 = Strongly Agree 
Descriptive Statistics for Commitmen t to Academic Excellence 


N Minimum Maximum Mean SD 


The quality of instruction in most of my Higher 
Education courses is/was excellent. 

There is a commitment to academic excellence 
in this program. 

Faculty provide timely feedback about my 
academic progress. 

There are a sufficient number of courses 
available each semester. 

The program has enhanced my intellectual 
development in the field of higher education. 

There are opportunities for me to evaluate the 
quality of instruction in the program. 


34 

1 

5 

4.12 

.946 

34 

1 

5 

4.00 

1.015 

34 

1 

5 

4.21 

.914 

34 

1 

5 

3.29 

1.142 

34 

2 

5 

4.26 

.751 

34 

2 

5 

4.24 

.699 
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The Higher Education program provides 


sufficient funding for assistantships, research 
studies, etc. 

34 

1 

5 

3.47 

1.308 

The Higher Education faculty encourages me as 

I progress through the program. 

33 

1 

5 

4.00 

1.000 

Scale: 1 = Strongly Disagree.. ,5=Strongly Agree 






Descriptive Statistics for Satisfaction with Research Courses 






N Minimum 

Maximum 

Mean 

SD 

Prepared me to conduct applied research to 

34 

1 


2.71 

1.292 

solve real-world problems in higher education. 

•J 

Prepared me to understand, analyze, synthesize, 
and critique relevant research when 
investigating current issues in higher education. 

34 

1 

5 

2.24 

1.075 


Prepared me to correctly design and execute all 
phases of a research study. 

34 

1 

5 

2.88 

1.274 

Prepared me to make appropriate decisions in a 
higher education setting based upon the results 
of current studies and research. 

34 

1 

5 

2.59 

1.258 

Prepared me to conduct collaborative research 

34 

1 

< 

2.59 

1.158 

with faculty and administrators. 

D 

Prepared me to present research study findings 
to an audience, such as at leadership/research 

34 

1 

5 

2.71 

1.315 


conferences. 

Scale: 1 = Strongly Disagree.. .5=Strongly Agree 
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